Learning Visual Localization of a Quadrotor Using Its Noise as Self-Supervision

We introduce an approach to train neural network models for visual object localization using a small training set, labeled with ground truth positions and large unlabeled one. assume that the be localized emits sound, which is perceived by microphone rigidly affixed camera. This information used as target of cross-modal pretext task: predicting sound features from camera frames. By solving task, model draws self-supervision audio data. The well suited robot learning: we instantiate it localize quadrotor 128 × 80 pixel images acquired robot. Experiments on separate testing set show introducing auxiliary task yields performance improvements: Mean Absolute Error (MAE) estimated image coordinates reduced 7 4 pixels; MAE distance 28 cm 14 cm. A has access labels entire 2 pixels 11 cm, respectively.